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Evaluation of PATB and Its Use for 
Selecting Personnel 

Summary of Major Findings and Recommendations 

Our assigned tasks were to review the evidence for 
validity and reliability of PATB and its use in the Agency 
for selecting personnel for professional positions, to 
assess it fairness for use with all applicants for profes- 
sional positions, and to make recommendations for improve- 
ment in the tests and their use. Our assignment did not 
include evaluation of the intensive psychological assessment 
procedures or any other procedures used to select personnel 
for jobs in the Agency, and these also require validation. 

Major Findings 

"X 

A. Use of PATB to Select Personnel 

1. If scores on PATB are used at_.al 1 to help select 
personnel, they enter the decision-making process 
only after a unit has indicated an interest in an 
applicant listed on the Applicant File Listing 
circulated by the Skills Bank. If no one indicates 
an interest in an applicant on this list within 
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Skills Bank. It no one indicates an interest i man 
applicant q el this 4d 4t--w4t4Lf n 10 days, then no 
report of that applicant's scores on PATB is made. 

2. Only about one-fourth of the supervisors in the 
units indicate that they give significant weight to 
the applicant's performance on PATB in making 
employment decisions. 

3. About two thirds of professional employees who 
entered on duty between 1 October 1977 and August 
1979 reported that they had taken PATB and one 
third reported that they had not. 

4* There appears to be no consistent policy among 
the units on requiring applicants to take PATB. 


B. Validity of PATB 

1. There is no explicit rationale for PATB and no 
evidence that the initial construction of the test 
battery was based on a systematic analysis of 
professional jobs in the Agency. • 

2. No studies have been done to establish the con- 



struct validity of the test s and scales that 
comprise PATB . 

There are no validity data of any kind for the 
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writing sample or for the Strong-Campbei 1 Interest 




Inventory. Although there have been some attempts 

— . — — 'V 

a 

to validate certain items on the Biographical 
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Information Inventory, evidence for its validit y 
ing. 
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is lackin g 

Four studies have been done on the validity of 
PATB for predicting success in foreign language 
training. The results of these studies indicate ,* (€ j 

very little or no value of PATB for this purpose. zs?** 
Out of 16 studies of the job related validity 
(criterion-related validity), only 10 met at 
least minimal standards for reporting data and 
could be used to judge the validity evidence. 

The evidence for job-related validity of PATB is 
fragmen tary, very weak and unconvincing. Correla- 
tions of test scores with supervisory ratings of 
job performance across similar jobs have been . 

op’hlir^— lyi&c — 

i nconsi stent and generally -low. Correlations of 

— - ? 

test scores with similar or identical criteria of 
job performance across jobs have also been incon- 
si stent and generally low. 
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7. Equations generated by multiple regression analysis 
or discriminant analysis to predict job performance « 
have been based on inadequate samples of small size 




STATIN TL 



and have not been cross-validated. — L 

8. The samples used for studying job-related validity 

and to generate equations. to„ predict job performance „ u 
't'Miave been composed^solely^pr predominantly of white 

these samples. There is no evidence that 

* 
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minorities have been included in these samples^ 
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9. ( The level of conf i denne with which the results «*,», tM&JL' ££'+ 

I ““ 

of these studies have been reported and used < ^zi,' s 





^to predict job performance of applicants greatly 
^£geds the level of confidence of the statistical 
vdata on validity and reliability. 

The evidence on validity presented in the 10 
studies does not meet minimum standards set 


by the American Psychological Association or 
the EEOC Uniform Guidelines on Employee Selec- 


tion Procedures (1978), 
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D. 


Reliability of PATB 

* 

1. No reliability data for any of the tests and 
scales of PATB are available for minorities. 

2. No reliability data for the work attitude scales 
are available for females. 

3. The reliability of the writing sample or of the 
scoring of the writing sample has not been deter- 
mined. 

4. For white males and females, only 5 tests--Reading 
Vocabulary, Reading Comprehension, Arithmetic 
Problems, Numerical Operations and Interpre- 
tation of Data--have reliabilities that are at 
least minimally acceptable for making decisions 
about individuals. For all of the other tests and 




scales, the reliabilities are too low; thus the 
scores from them are too unstable and inaccurate 
for use in making decisions about individuals. 
The Narrative Report — 

1. No written guidelines are used by the psycho! 

/ 

gists in writing the narrative report and the 
reports tend to vary considerably in unpredict- 
able ways. The variations appear to be due as 
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3. 


much or more to the personal idiosyncrasies of 
the psychologists as to differences in performance 
on PATB among applicants. 

The strong, confident recommendations to hire 
Or not hire applicants for specific jobs or in 
specific units are not supported by the availa- 
ble evidence on validity of PATB. 

The scores from the Strong-Campbell Interest 







Inventory are being consistently misinterpreted^) '**~**~ p tp f 


4. 


in the narrative reports. 

The reports of writing ability use ambiguous 
descriptors, are noncomparable from one report 
to another, and are inconsistent in the aspects 
of writing ability reported on. 

E. Evidence of Unfairness 


1. 


There is no evidence that studies of adverse 
impact as defined by EEOC have been done for 
PATB or for 4Dy Other s el ect-io n p rocedur e used 1n_ 
the Agency . 
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Although we could find no direct evidence on 
bias or unfairness, the following factors indi- 
cate that there is ^serious) potential for /misuse 





Or unfair use of PATB. 
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a. The re is no evidence that minorities 
were represented in the groups used to 
norm the tests. 

b. There is no evidence that minorities 
have been included in the samples used 
to determine job-related validity. There 
is evidence that females have not been 
represented at all in some samples and 
are underrepresented in other samples. 

C. The equations being used to make recommen- 
dations for hiring employees have not 
been cross-validated. The sample used 
to generate these equations appears not 
to have included any minorities and to 
have had very few females. 

F. Other Findings 

1. Units in the Agency are using other tests con- 



-ykh " + structed within the units to select personnel. 

5 ^ 


There is no evidence that these tests have been 
/ rU*- ct f / ^ V ' validated. 
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The tests and scales of PATB are over 20 years 
old. The content of some of the tests is dated 
and needs revision. 
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Recommendati ons 


We recommend that: 

1 . A systematic analysis of professional jobs in the\^ 
.Agency be done to determine the abilities needed to perform 
each job. On the basis of this analysis, it should be 
determined whether there is enough commonality in job 
requirements and demands to justify a single selection 
battery for use with applicants for professional jobs. If 
the answer is yes, and we think it will be, then a new 
battery should be constructed or the present PATB should be 
revised to appraise the abilities brought out by the job 
analyses. 

2. Policy guidelines be set as to which applicants^ 
for professional jobs should be required to take PATB.jf 

3. The use of tests constructed in units but 
have not been validated be controlled. 

4. Equations to predict job performance not be used 
for minorities and women until both of these groups have 
been adequately represented in the samples used to generate 
the equations. 

5. Equations to predict job performance for white 
males not be used until they have been cross-validated. 

6. The use of the present narrative report prepared 
by psychologists be discontinued and instead a profile 
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report or a completely standardized verbal description of 
performance be generated by ‘computer. 


T 



7. The self-report instruments such as work attitude 
scales, temperament scales and biographical information not 
be use d as a part of the basic battery to select employees. 
They are subject to faking, call upon the individual to 
testify against himself", lack demonstrated validity for 
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performance of jobs, and have very low reliabilities.' TfuuA ^ 
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8. The use of the Strong-Campbell Interest Inventory o-Ktv- 

be discontinued because it lacks Agency norms and Agency 
validation, and is being c onsiste ntly misi nterprete d in the Y 
narrative reports. ^ 

9. Test results enter more actively into the selection 
process and the Applicant File Listing call attention 
to those applicants with superior scores on the cognitive 
tests. 


v * 


ID. Improved procedures for storage and retrieval 
of applicants' files be developed so that records of prom- 
ising applicants, especially those with high abilities or 
special skills, can be considered for employment. 

If no job analysis is done and the Agency decides to 
continue with the cognitive test portions of PATB, we 
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think the resultant "PATB" should be revised to improve its 

at 

reliability and validity. We recommend that: 

1. The Numerical Operations test be dropped. It i 

appraising speed of simple arithmetical computations which^ 

\-bejsiks-4* *n 


does not seem to be related to Agency jobs. 

The Considerations test be dropped because it has 
very low reliability, needs to be hand-scored, and is not 



i* Tt r i 

^ ^ measuring an ability that appears relevant to Agency jobs. 

Ip terpretation °f Data test be dropped because 
content is dated. In its place, a new test be devised 
which would appraise the ability to synthesize information 
from separate and inconsistent sources. 

4. The Vocabulary test be revised to exclude exotic 
words and to relate its content more closely to Agency 
jobs. 
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5. The Reading Comprehension test be revised re- 
placing the literary reading selections with reading 
passages drawn from materials that must-be read and compre- 
hended by people performing Agency jobs. 

6. The Arithmetic Problems test be revised replacing 
dated items. 

7. The writing task be modified to make it corres- 
pond more closely to the type of writing done in profes- 
sional jobs in the Agency. 
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8;,. The battery of tests be renormed on a represen- 

» 

tati've sample of applicants making sure that minorities have 
the same proportional representation in the norm sample as 
they have in the applicant sample. 
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